Fix memory leak in stats collector #3069
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This addresses #2865 where a goroutine leak was slowly increasing memory consumption.
Implementation details
dockerGoClient.Stats
creates a<-chan *types.StatsJSON
which is never closed, therefore the send operationstatsC <- data
blocks for ever, which in turn keeps the corresponding goroutine alive for ever.The solution is to simply do a send operation, but returning if the context (used to invoke
Stats
) is cancelled. This ends the offending goroutine.i.e.
Testing
Test in a long-lived cluster and verify memory is no longer increasing indefinitely.
New tests cover the changes: yes
Description for the changelog
Bug - Fixes #2865 caused by a memory leak in stats collector.
Licensing
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.